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IN THE U.S. PATENT AND TRADEMARK OFFICE 

Applicant: Min-Cheol Hong et al . 

Appl. No.: New Group: Unassigned 

Filed: October 20, 2000 Examiner: UNASSIGNED 

For: FILTERING CONTROL METHOD FOR IMPROVING 

IMAGE QUALITY OF BI-LINEAR INTERPOLATED 
IMAGE 

PRELIMINARY AMENDMENT 

Assistant Commissioner for Patents October 20, 2000 

Washington, DC 20231 

Sir: 

The following preliminary amendments and remarks are 
respectfully submitted in connection with the above-identified 
application . 

IN THE CLAIMS : 

Please amend the claims as follows: 



Claim 



Line 2, delete "or 6' 



Claim 15 



Line 2, delete "or claim 13' 



Appl. No. New 

Please add the following new claims: 

--17. The filtering control method for improving the 

image quality of the bi-linear interpolated image according to 
claim 6, the number of a kernal of the PSF(P) is set in 
accordance with an up-sampling value of the image. - 

--18. The filtering control method for improving the 

image quality of the bi-linear interpolated image according to 
claim 13, the number of a kernal of the PSF(P) is differently 
set in accordance with an up-sampling value of the image. — 

REMARKS 

Claims 1-18 are pending in the present application. Claims 
8 and 15 have been amended to remove multiple dependencies. 
Claims 17 and 18 have been added to recapture subject matter 
that was deleted from claims 8 and 15. 

Entry of the above amendments is earnestly solicited. An 
early and favorable first action on the merits is earnestly 
solicited. 

Should there be any outstanding matters that need to be 
resolved in the present application, the Examiner is 
respectfully requested to contact Gary D. Yacura (Reg. 35,416) 
at the telephone number of the undersigned below. 
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FILTERING CONTROL METHOD FOR IMPROVING IMAGE QUALITY OF BI- 
LINEAR INTERPOLATED IMAGE 



BACKGROUND OF THE INVENTION 

5 

1. Field of the Invention 

The present invention relates to an interpolation method adapted to 
enlargement of a low resolution image when the image digitized through a CCD 
(Charged-Coupled Device) has the low resolution, in particular to a filtering control 
10 method for improving the image quality of a bi-linear interpolated image which is 
capable of restoring a requested interpolated high resolution image from a low 
resolution image by finding a coefficient of a two-dimensional filter on the basis of 
a regularization image restoration method. 

15 2. Description of the Prior Art 

In the conventional technology, a still picture or a moving picture has or 
transmits a low resolution image because it can not physically satisfy a sensor 
having the low resolution or a nyquist value. 

In addition, a compressed moving picture has or transmits the low 
20 resolution image due to its bit value problem. 

For example, when the compressed moving picture having the low bit 
value is transmitted to a receiver and the receiver enlarges the transmitted moving 
picture, the resolution of the transmitted moving picture lowers due to a 
degradation phenomenon ect.. 
25 Accordingly, a method for getting a high resolution image from a low 



r 



resolution image is required. 

In the meantime, the method for getting the high resolution image from the 
low resolution image is largely divided into an image expansion type method and 
an image enhancement type method. 
5 First, the image expansion type method converts the size of the low 

resolution image into a requested size. The bi-linear interpolation method, a zero 
order expansion method, and a cubic spline method are comprised in the image 
expansion type method. 

However, as described above, the image expansion type method has an 
10 image visibility lowering problem because when the image is interpolation-restored 
by the above-mentioned method such as the bi-linear interpolation method, zero 
order hold expansion method, cubic spline method, the outlines of the image is 
over-blurred. 

Meanwhile, the image enhancement type method comprises many 
15 methods, but the image enhancement type method causes a computational 
complexity, accordingly the method is not suited to a real-time processing due to 
the its computational complexity. 

In addition, when the image enhancement type method is used for getting 
the high resolution image from the low resolution image, setting of each parameter 
20 is not adaptable. 

For example, there is a POCS (Projection Onto Convex Set) method for 
increasing the resolution of an image. In the POCS method, in use of time region 
information, it is assumed as correlation between the images is uniformly same, 
but actually the correlation between the images is not uniform. 
25 In addition, there is a mapping method for mapping a non-uniform sample 



of the low resolution image into a uniform sample of the high resolution image by 
using moving information and segmentation information of the image. However, 
the mapping method has the computational complexity problem, accordingly the 
mapping method is not suited to the real-time image data processing of the image 
5 processing system. 

SUMMARY OF THE INVENTION 

The object of the present invention is to provide a filtering control method 
10 for improving the image quality of a bi-linear interpolated image which is capable 
of improving the image quality of the interpolated image by using an interpolation 
method considering a real-time processing, a computational complexity and an 
efficiency when the digital video system seeks the interpolated image from the low 
resolution image. 

15 The other object of the present invention is to provide the filtering control 

method for improving the image quality of the bi-linear interpolated image which is 
capable of finding a two-dimensional filter coefficient for getting the interpolated 
image from the low resolution image on the basis of a regularization image 
restoration method. 

20 The other object of the present invention is to provide the filtering control 

method for improving the image quality of the bi-linear interpolated image which 
can approximate and find a PSF (Point Spread Function) for the bi-linear 
interpolated image from a modeling of the degraded image in the frequency region. 
The other object of the present invention is to provide the filtering control 

25 method for improving the image quality of the bi-linear interpolated image which is 



capable of performing a real-time adaptive processing by finding a filter coefficient 
from the bi-linear interpolated image and approximated PSF. 

In the present invention, in order to find a filter coefficient for finding the 
interpolated image from the low resolution image on the basis of the regularization 
5 image restoration method, when H is the PSF (Point Spread Function), f is a 
requested high resolution image, Z is the low resolution image, g is the high 
resolution image gotten from the bi-linear interpolation method, an added function 
M (f) = || g-Hf || 2 +a || Cf || 2 for finding the PSF(H) from an equation g=Bz=Hf+n 
(B, H are bi-linear interpolated filters, n is a noise component generated by the 
10 assumed H) is defined. 

The filtering control method for improving the image quality of the bi-linear 
interpolated image can be implemented by finding the PSF(H) from the added 

function M(f) by using an equation H(k,l)= G ^ k ^ . 

F(k,l) 

The filtering control method for improving the image quality of the bi-linear 
15 interpolated image can be implemented by finding a PSF(P) of a f=Pg function by 

H*(k I) 

using an equation P( k,l ) = after finding the 

y M H*(k,l)H(k,l) + C*(k,!)C(kJ) 

PSF(H). 

The filtering control method for improving the image quality of the bi-linear 
interpolated image can restore the requested high resolution image(f) by finding an 
20 added filter coefficient Q of the PSF(P) and interpolation filter B from the equation 
f=Pg=PBz=Qz. 

BRIEF DESCRIPTION OF THE DRAWINGS 
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FIG.1 illustrates an image sample for getting a twice enlarged high 
resolution image according to the embodiment of the present invention. 

FIG. 2 illustrates an interpolation filter coefficient for getting the twice 
5 enlarged image according to the embodiment of the present invention. 

FIG. 3 illustrates an image sample for getting a three times enlarged high 
resolution image according to the other embodiment of the present invention. 

FIG.4 illustrates the interpolation filter coefficient for getting the three times 
enlarged image according to the other embodiment of the present invention. 
io FIG. 5 illustrates an image sample for getting a six times enlarged high 

resolution image according to the another embodiment of the present invention. 

FIG. 6 illustrates the interpolation filter coefficient for getting the six times 
enlarged image according to the another embodiment of the present invention. 

15 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

FIG.1 illustrates an image sample for getting a twice enlarged high 
resolution image according to the embodiment of the present invention. 

As depicted in FIG.1, a ~ i illustrate low resolution pixels, A~D illustrate 
20 high resolution pixels. In addition, pixels depicted as 'x' illustrate pixels interpolated 
as twice by a twice interpolation filter coefficient. 

FIG. 2 illustrates the interpolation filter coefficient for getting a twice 
enlarged image according to the embodiment of the present invention. In other 
words, the interpolation filter coefficient for interpolating the twice enlarged image 
25 of FIG.1 is depicted in FIG. 2. 

5 



As depicted in FIG. 2, the high resolution image is gotten from the low 
resolution pixels a~i (3x3 pixels) inside of a circle of FIG.1 by using the 
interpolation filter coefficient. 

FIG. 3 illustrates an image sample for getting a three times enlarged high 
5 resolution image according to the other embodiment of the present invention. 

As depicted in FIG. 3, a~p illustrate the low resolution pixels, A~l illustrate 
the high resolution pixels using the filter according to the present invention. 

FIG.4 illustrates the interpolation filter coefficient for getting the three times 
enlarged image according to the other embodiment of the present invention. 
io As depicted in FIG.4, three times enlarged pixels which are newly 

generated illustrated as triangles in FIG. 3 are gotten from the low resolution pixels 
a~p (4x4 pixels) by using the interpolation filter coefficient of FIG.4. 

FIG. 5 illustrates the image sample for getting a six times enlarged high 
resolution image according to the another embodiment of the present invention. In 
15 other words, it illustrates the image sample for getting the six times enlarged high 
resolution image from the twice and three times interpolation filter coefficients by 
using the bi-linear interpolation method. 

As depicted in FIG. 5, pixels illustrated as a 'X' can be gotten by using the 
twice interpolation filter of FIG. 2, and pixels illustrated as a triangle can be gotten 
20 by using the three times interpolation filter coefficient of FIG.4. 

In addition, pixels illustrated as a quadrilateral can be gotten from the 
pixels generated by the twice and three times interpolation filter coefficients by 
using the bi-linear interpolation method. 

FIG. 6 illustrates the interpolation filter coefficient for getting the six times 
25 enlarged image according to the another embodiment of the present invention. In 

6 



other words, the interpolation filter coefficient for getting the six times enlarged 
image of FIG. 5 is depicted in FIG. 6. 

Meanwhile, as depicted in FIG. 2, FIG.4 and FIG. 6, the value found by 
using the interpolation filter coefficient of the present invention has an integer 
5 value. 

In addition, a 9bit shift is performed to the value calculated by the 
interpolation filter coefficient, accordingly there is no need to perform a floating 
point operation processing. 

The twice, three times, six times interpolated images are depicted in FIG.1 
10 ~ FIG. 6, however the present invention is not limited by that, it can be adapted 
freely to a certain interpolation value. 

Hereinafter, the filtering control method for improving the image quality of 
the bi-linear interpolated image will be described in more detail. 

First, a spatially invariant PSF (Point Spread Function) for finding the 
15 interpolation filter coefficient according to the each interpolation value can be 
easily analyzed and approximated in the frequency region, accordingly the 
spatially invariant PSF (Point Spread Function) is considered from the bi-linear 
interpolated image. 

After that, when it is assumed as the low resolution image is z, high 
20 resolution image gotten by the bi-linear interpolation method is g, high resolution 
image to be restored is f, the relation between the each image can be described 
as below. 

[Equation 1] 
25 g=Bz=Hf+n 
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Herein, the B, H, n are the bi-linear interpolation filters, H is the spatially 
invariant PSF defining the relation between the original high resolution image and 
high resolution image gotten by the interpolation method, and the n is a noise 
5 component generated by the assumed H. 

Herein, when the noise component is neglected and a direct inverse is 
used in order to find the PSF(H), the PSF(H) can be described as below equation 
2 in the frequency region. 



10 [Equation 2] 

F(k,l) 



Herein, the H{k,l) is the component in the k,l frequency region of the 
PSF(H), the G (kj) is the component in the k,l frequency region of the bi-linear 
15 interpolated image. In addition, the F (kj) is the component in the kj frequency 
region of the high resolution image. 

Meanwhile, the high resolution image f to be restored is unknown, the 
PSF(H) can be gotten from the bi-linear interpolated high resolution image through 
a statistical processing after performing an under-sample processing of various 
20 images as various value. 

Herein, the high resolution image is gotten by using the PSF(H) found from 
the direct inverse. In other words, there is a system stabilization problem because 
the high resolution image gotten from the PSF(H) by using the direct inverse is 
overshoot in the region where the k,l have '0' value (in general, high frequency 

8 



region) in the frequency region, accordingly the regularization image restoration for 
improving the system stabilization is used to solve the problem. 

The regularization image restoration method is used for restoring the 
image or finding a certain PSF, an added function M(f) for finding the PSF(H) by 
5 using the regularization image restoration method can be described as below 
equation 3. 

[Equation 3] 

M(f) = |jg-Hf || 2 +a || Cf || 2 

10 

Herein, the first term of the right side of Equation 3 illustrates the credibility 
of the bi-linear interpolated image, the second term of the right side illustrates 
increase of the stability of the system by providing the mitigation to the restored 
image. 

15 In addition, the || . |[ means a norm, the a is a regularization parameter 

for determining the credibility and mitigation of the original image. In addition, the 
C is the two-dimensional high frequency filter for determining the mitigation of the 
original image, in the present invention a two-dimensional Gaussian filter is used 
as the C. 

20 When a gradient operator is adapted to Equation 3 in order to get the high 

resolution image, it can be described as below equation 4. 

[Equation 4] 

□ f M(f) = -2H T (g-Hf)+2 a C T Cf=0 

25 
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Herein, the T means a transpose of a matrix. 

Meanwhile, conventionally a repetition method is used in order to get the 
high resolution image and regularization parameter, but it is not suited to the 
moving picture processing because the method causes lots of computational 
5 complexity. 

Accordingly, in the present invention, the regularization parameter a is 
fixed as '1', and the high resolution image f can be found as below equation 5. 



[Equation 5] 



io f= r H g T =Pg 

{H T H + C T C) 



When the PSF(P) is found by Equation 5, PSF(P)=H/(H T H+C T C) requires 
the lots of computational complexity for calculating an inverse matrix, however the 
PSF(P) in Equation 5 is a block-circulant matrix, accordingly it can be easily 
15 calculated in the frequency region. 

Accordingly, the PSF(P) can be found finally as below Equation 6. 



[Equation 6] 

H*(k,l) 



P(k,l)= 



H * (k, l)H(k, I) + C * (k, I)C(k, I) 
20 

Herein, the '*' means a complex-conjugate. 

The PSF(P) can be found by using an IFT (Inverse Fourier Transform) 
from Equation 6. 

10 



The requested high resolution image f can be found as below Equation 7 
by using the found PSF(P) and Equation 1 . 

[Equation 7] 
5 f=Pg=PBz=Qz 

The PSF(P) is the spatially invariant function, the bi-linear interpolation 
filter B can be easily found by the conventional technology, accordingly the added 
filter coefficient Q of the PSF(P) and bi-linear interpolation filter B can be found. 
io Herein, in order to reduce the computational complexity, the number of a 

kernel of the PSF(P) is set in accordance with the up-sampling value. 

When the up-sampling value is 2 in the present invention, the number of 
the kernel is limited as 3, when the up-sampling value is 3, the number of the 
kernel is limited as 4. 

15 When the up-sampling value is 2, it can be used in an application segment 

for enlarging the size of the image as twice at a post processor of the compressed 
digital image and in finding of a sub-pixel moving vector in a H.263 moving picture 
compressed method. 

In addition, when the up-sampling value is 3, it can be used in using of a 
20 1/3 unit moving vector in a H.26L moving picture compressed method. 

Herein, the H.263 and H.26L are moving picture compressed standards 
presented in the ITU-T (International Telecommunications Union- 
Telecommunication). 

As described above, the present invention can be used for improving the 
25 image quality at the post processor of the compressed digital image by using the 

li 



interpolation method for getting the interpolated high resolution image from the low 
resolution image when the resolution of the digital image lowers due to the low 
resolution image sensor. 

In addition, the interpolation method of the present invention can improve 
5 the image quality by finding the moving vector of the moving picture compressed 
type. 
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What is claimed is : 

1 A filtering control method for improving the image quality of a bi- 
5 linear interpolated image in methods for getting a high resolution image from a low 
resolution image, comprising : 

restoring a requested high resolution image f by finding an added filter 
coefficient Q of a PSF(P) and a bi-linear interpolation filter B from an equation 
f=Pg=PBz=Qz, herein the f is the high resolution image as requested, P is the PSF 
10 (Point Spread Function), g is the high resolution image found by the bi-linear 
interpolation method, and z is the low resolution image. 

2. The filtering control method for improving the image quality of the 
bi-linear interpolated image according to claim 1, wherein the high resolution 
15 image f can be restored by performing an added function M(f) definition process 
for finding the PSF(H) from an equation g = Bz = Hf+n, herein the B, H are bi- 
linear interpolation filters, and the n is a noise component generated by the 
assumed H. 

20 3. The filtering control method for improving the image quality of the 

bi-linear interpolated image according to claim 1, wherein the high resolution 
image f is restored by finding a PSF(P) of a f=Pg function after finding the PSF(H) 
from the added function M(f). 

25 4. The filtering control method for improving the image quality of the 



13 



bi-linear interpolated image according to claim 2, wherein the added function M(f) 
is defined as M(f)= J| g-Hf || 2 +a || Cf || 2 , herein the a is a regularization parameter, 
C is a two-dimensional high frequency filter for finding mitigation of the original 
image. 

5 

5. The filtering control method for improving the image quality of the 
bi-linear interpolated image according to claim 3, wherein the PSF(H) is found by 

using an equation H(k,l)= G ^ k ^ i herein the G(k,l) is the component in the k,l 

F(k,l) 

frequency region of the bi-linear interpolated image, and the F( kj ) is the 
10 component in the kj frequency region of the high resolution image. 



6. The filtering control method for improving the image quality of the 
bi-linear interpolated image according to claim 1, wherein the PSF(P) can be found 
by getting an I FT (Inverse Fourier Transform) by an equation 

H*(k,l) 



15 P(k, l)= 



H * (k,l)H{k,l) + C * (k,l)C(k,l) 



7. The filtering control method for improving the image quality of the 
bi-linear interpolated image according to claim 4, wherein the regularization 
parameter a is fixed as '1' in order to reduce a computational complexity. 

8. The filtering control method for improving the image quality of the 
bi-linear interpolated image according to claim 1 or 6, the number of a kernel of the 
PSF(P) is set in accordance with an up-sampling value of the image. 

14 
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9. The filtering control method for improving image quality of the bi- 
linear interpolated image according to claim 4, wherein a two-dimensional 
gaussian filter is used as the two-dimensional high frequency filter C in order to 

5 determine the mitigation of the original image. 

10. A filtering control method for improving image quality of a bi-linear 
interpolated image in methods for getting a high resolution image from a low 
resolution image, comprising : 

10 defining an added function M(f) for finding a PSF(H) from an equation 

g=Bz=Hf+n (B, H are bi-linear filters, N is a noise component generated by an 
assumed H when the H is a PSF (Point Spread Function), F is a requested high 
resolution image, z is a low resolution image, and g is a high resolution image 
gotten by the bi-linear interpolation method ; 

15 finding a PSF(P) of a f=Pg function after finding the PSF(H) from the 

defined added function M(f) ; and 

restoring the requested high resolution image f by finding an added filter 
coefficient Q of the PSF(P) and interpolation filter B from the equation 
f=Pg=PBz=Qz. 

20 

11. The filtering control method for improving the image quality of the 
bi-linear interpolated image according to claim 10, wherein the added function M(f) 
is defined as M(f)= j| g-Hf || 2 +a || Cf || 2 , herein the a is a regularization parameter, 
and C is a two-dimensional high frequency filter for finding the mitigation of the 

25 original image. 

15 



12. The filtering control method for improving the image quality of the 
bi-linear interpolated image according to claim 10, the PSF(H) is found by an 

equation H(k,l)= G ^ k ^ _ herein the G(k,l) is the component in the kj frequency 

F{k,l) 

5 region of the bi-linear interpolated image, and the F(k,l) is the component in the 
k,l frequency region of the high resolution image. 



13. The filtering control method for improving the image quality of the 
bi-linear interpolated image according to claim 10, wherein the PSF(P) is found by 
10 using an I FT (Inverse Fourier Transform) from an equation 

H*(k,l) 



P(k,l)= 



H * (k, l)H(k, I) + C * (k, l)C(k, 1) 



14. The filtering control method for improving the image quality of the 
bi-linear interpolated image according to claim 11, wherein the regularization 
15 parameter a is fixed as '1' in order to reduce a computational complexity. 



15. The filtering control method for improving the image quality of the 
bi-linear interpolated image according to claim 10 or claim 13, the number of a 
kernel of the PSF(P) is differently set in accordance with an up-sampling value of 
20 the image. 



16. The filtering control method for improving image quality of the bi- 
linear interpolated image according to claim 11, wherein a two-dimensional 

16 
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gaussian filter is used as the two-dimensional high frequency filter C in order to 
determine the mitigation of the original image. 



25 
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ABSTRACT OF THE DISCLOSURE 

The present invention relates to an interpolation method for enlarging a 
5 digital image or predicting a moving vector of a compressed image system as a 
sub-pixel unit when the image digitized through a CCD (Charge Coupled Device) 
camera ect. has a low resolution in a video phone or video conference or general 
digital video system, particularly the present invention can be adapted to a post 
processor of a compressed digital image in order to improve the image quality, and 
10 can be used for finding a moving vector of a moving picture compressed type, 
accordingly the present invention is capable of improving the image quality. 
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